In the past, people met face-to-face when they wanted to communicate. But, in today's mobile, widely dispersed, and increasingly interconnected society, people often need to communicate with others who are far away. In order to facilitate this communication, video conferencing is gaining in popularity.
In video conferencing, both parties have a conferencing system that may include a microphone, a camera, a speaker, and a video monitor, and the parties are connected to each other via a network. In this way, the parties can converse while viewing moving video images of each other. Video images contain a large amount of data, which requires a large amount of storage and data transfer bandwidth, so the sending conferencing system often compresses the video and the receiving system decompresses the video prior to display.
Video compression algorithms use interframe compression to remove temporal redundancies between frames. Interframe compression involves storing only the differences between successive frames in the data stream. Interframe compression stores the entire image of a key frame or reference frame, generally in a moderately compressed format. Then, successive frames are compared with the key frame, and only the differences between the key frame and the successive frames are stored and transmitted. Periodically, such as when new scenes are displayed, new key frames are stored, and subsequent comparisons begin from this new reference point. Examples of video compression that use an interframe compression technique are MPEG (Moving Picture Experts Group), H261, H263, H264 (also known as JVT (Joint Video Team), AVC (Advanced Video Coding), MPEG 4 part 10), DVI (Digital Video Interactive) PLV (Production Level Video), and Indeo, among others. The H264 standard is well suited to an embodiment of the invention since it defines multiple references frames. The video decoding process is generally the inverse of the video encoding process and is employed to reconstruct a moving image sequence from a compressed and encoded bitstream. The receiver decodes data in the bitstream according to a syntax that is defined by the data compression algorithm.
Interframe compression works well as long as the receiver receives all of the data packets. Unfortunately, some networks, such as the Internet, suffer from a high rate of packet loss and resulting transmission delays. In particular, depending on conditions such as how congested the Internet is at any given time, loss of entire packets has been found to occur on the Internet at a rate of up to 25%, or up to one in every four packets.
When the receiver detects a packet loss, the receiver either requests transmission of a keyframe or a portion of a keyframe or requests retransmission of the lost packet(s). Thus, the reconstruction and display of the video must wait until the retransmitted packet(s) have been received. Real-time video signals (especially compressed signals) are highly sensitive to delay and will appear jumpy, interrupted, or otherwise distorted if the packets do not flow continuously to the receiving end. Therefore, although the loss of packets in a real time video transmission has been correctable, the resulting video images have often been of unacceptable quality, leading to user dissatisfaction.
Although the problems of packet loss have been described in the context of video conferencing, they can also occur with transmission of video data over the Internet or with transmission of any compressed data.